Skip to content

9ff3f391: reconcile upload accepts Finom CSV statements - #342

Merged
alexeygrigorev merged 1 commit into
mainfrom
fix/reconcile-finom-statement
Nov 5, 2026
Merged

alexeygrigorev merged 1 commit into
mainfrom
fix/reconcile-finom-statement

Conversation

@alexeygrigorev

Copy link
Copy Markdown
Member

Task 9ff3f391 — "CLI gap: reconcile upload Finom statement".

Problem

The reconcile flow parses Finom CSV exports only through the email intake (recognizeEmailStatement, backend/src/reconcile/statementIntake.ts). Every explicit upload layer was PDF-only, so a Finom statement CSV had no way into reconciliation except emailing it to the invoice address:

  • CLI contentTypeFor mapped only .pdf (cli/src/documentUpload.mjs)
  • dataops reconcile upload ran requirePdfBytes unconditionally (cli/src/commands/reconcile.mjs)
  • document prepare refused non-PDF statements (backend/src/routes/bookkeepingDocuments.ts)
  • completion verified the PDF signature (backend/src/routes/bookkeepingShared.ts)
  • recognizeStatement accepted PDFs only (backend/src/reconcile/statementIntake.ts), pinned by tests

Change

  • Backend recognition: recognizeStatement now recognizes tabular exports from their header row against the same fixed layouts the email intake uses (FIXED_LAYOUTS.finom: Completed date, Counterparty name, Reference, Amount, Balance — already in repo with a synthetic fixture) plus Revolut, with the selected-account mismatch check. The email intake path is behavior-identical (it now calls recognizeStatement directly; errors carry the filename).
  • Identity rule: a Finom CSV prints no IBAN, so the parsed rows have no bank-account identity. prepareStatementImport commits such a statement only when the operator explicitly selected the account (--account finom), which stands in for the header identity; without it the source stays at review exactly as the email intake does (StatementIdentityReviewError). Rows key as unverified:<statementId> per the existing selection machinery, so dedup on re-upload follows the repo's existing conventions: same file replays to the same statement id (statementIdFor(documentId, account, period), preview says "no differences" and requires --replace); overlapping rows from a different file are kept once (earliest upload wins) by rowKeys (date+amount+currency+counterparty+reference, occurrence-numbered). Only a statement that has header identity refuses to coexist with an unverified overlapping source (clearer message now).
  • Document storage: bank-statement / private-account-statement documents accept text/csv at prepare and completion; the object must be readable text (no NUL bytes, no PDF signature mismatch). XLSX and every other document type stay PDF-only.
  • CLI: dataops reconcile upload --file statement.pdf|statement.csv (upload-statement registered as an alias) validates bytes locally, stores the document and imports it; help documents the CSV/--account rule.

Tests (targeted only, all synthetic)

  • backend/tests/reconcile.test.ts: Finom CSV import end to end on an isolated synthetic month (2029-10) — review without account, read-only parse, parse with wrong account, import with account (7 rows, EUR closing balance), idempotent replay (preview identical → --replace keeps the same statement id and rowCount), balance lookup, deduped overlapping second export, unknown-CSV refusal, legacy CSV-backed rows readable.
  • backend/tests/bookkeeping-document-atomic-api.test.ts: text/csv prepare+complete succeeds; XLSX still refused at prepare; declared-PDF object with CSV bytes still refused at completion.
  • cli/test/reconcile.test.mjs: CSV upload passes text/csv through prepare and imports under the named account; XLSX / PDF-mislabeled / binary-.csv / receipt-.csv refusals keep usage exit codes.

Results

  • backend: reconcile.test.ts 11/11, bookkeeping-document-atomic-api + reconcile-email-intake + reconcile-parsers + reconcile-statement-selection 36/36, reconcile-month/-service/-automatch/-backtest/pdf-layouts/platform-payouts/bookkeeping-auto-match 73/73, api-bookkeeping + invoice-finance-extraction + invoice-replace-document + bookkeeping-audit 53/53
  • cli: reconcile.test.mjs 30/30, bookkeeping + cli 45/45
  • backend tsc --noEmit clean

Read-only toward prod; no real financial data (fixtures are synthetic, invented counterparties). Frontend portal upload stays PDF-only (unchanged) — a portal CSV flow can follow the same backend path later.

The reconcile flow could parse Finom CSV exports only through the email
intake; the CLI, portal API and document upload path were PDF-only at
every layer, so a Finom statement CSV had no way into reconciliation.

- recognizeStatement now recognizes tabular exports from their header
  row against the same fixed layouts the email intake already uses
  (Finom statement, Revolut), including the selected-account mismatch
  check; the email intake keeps its behavior, with errors that now
  carry the filename.
- prepareStatementImport commits an identity-less tabular statement
  only when the operator explicitly selected the account (--account),
  which stands in for the missing IBAN; without it the source stays at
  review, as before. Only a source with header identity refuses to
  coexist with an unverified overlapping source, so re-uploads of the
  same file replay to the same statement id.
- Bank-statement documents accept text/csv at prepare and completion
  (the object must be readable text, not a renamed PDF or binary);
  XLSX and every other document type stay PDF-only.
- dataops reconcile upload (alias: upload-statement) accepts
  statement.pdf or statement.csv, validating bytes locally before any
  upload; help documents the --account rule for CSV exports.

Scope: in
@alexeygrigorev
alexeygrigorev merged commit 503c9e5 into main Nov 5, 2026
4 checks passed
@alexeygrigorev
alexeygrigorev deleted the fix/reconcile-finom-statement branch November 5, 2026 15:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant